Papers with training paradigms
Comprehensive Multi-Dataset Evaluation of Reading Comprehension (D19-58)
Copied to clipboard
| Challenge: | Recent research aims to facilitate training and evaluation on several reading comprehension datasets at the same time. |
| Approach: | They propose an evaluation server that reports performance on seven diverse reading comprehension datasets and includes synthetic augmentations to test models' ability to handle out-of-domain questions. |
| Outcome: | The evaluation server performs on seven reading comprehension datasets, and collects and includes synthetic augmentations for these datasets to test models' ability to handle out-of-domain questions. |
MathMixup: Boosting LLM Mathematical Reasoning with Difficulty-Controllable Data Synthesis and Curriculum Learning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing data synthesis methods suffer from limited diversity and lack precise control over problem difficulty, making them insufficient for efficient training paradigms such as curriculum learning. |
| Approach: | They propose a data synthesis paradigm that generates high-quality, difficulty-controllable mathematical reasoning problems through hybrid and decomposed strategies. |
| Outcome: | The proposed paradigm outperforms existing methods and improves mathematical reasoning abilities. |
Conformity in Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Conformity is a form of social influence that affects the way people respond to information. |
| Approach: | They adapt psychological experiments to examine the extent of conformity in large language models. |
| Outcome: | The proposed interventions mitigate conformity by reducing the naturalness of majority tones and reducing instruction-tuned models. |
Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs (2022.emnlp-main)
Copied to clipboard
| Challenge: | We show that one of today’s largest language models lacks this kind of social intelligence out-of-the-box, using two tasks: SocialIQa and ToMi. |
| Approach: | They propose to use social intelligence and Theory of Mind to examine whether modern large-scale language models lack this kind of social intelligence out-of-the-box. |
| Outcome: | The proposed model lacks social intelligence out-of-the-box, and has well-below human accuracies on SocialIQa and ToMi, respectively. |
ChatVLA: Unified Multimodal Understanding and Robot Control with Vision-Language-Action Model (2025.emnlp-main)
Copied to clipboard
Zhongyi Zhou, Yichen Zhu, Minjie Zhu, Junjie Wen, Ning Liu, Zhiyuan Xu, Weibin Meng, Yaxin Peng, Chaomin Shen, Feifei Feng, Yi Xu
| Challenge: | Recent advances in vision-language-action models prioritize robotic action mastery . however, models trained on visual-text pairs struggle to interpret multimodal data . |
| Approach: | They propose a framework that integrates multimodal data after initial control mastery and a Mixture-of-Experts architecture to minimize task interference. |
| Outcome: | The proposed framework surpasses state-of-the-art vision-language-action (VLA) methods on multimodal understanding benchmarks and achieves six times higher performance on visual question-answering datasets. |
CoLo: A Contrastive Learning Based Re-ranking Framework for One-Stage Summarization (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods for extractive and abstractive summarization use token-level or sentence-level training objectives. |
| Approach: | They propose a Contrastive Learning based re-ranking framework for one-stage summarization called CoLo. |
| Outcome: | The proposed framework boosts extractive and abstractive results on CNN/DailyMail benchmarks while maintaining inference efficiency. |
AnyGPT: Unified Multimodal LLM with Discrete Sequence Modeling (2024.acl-long)
Copied to clipboard
Jun Zhan, Junqi Dai, Jiasheng Ye, Yunhua Zhou, Dong Zhang, Zhigeng Liu, Xin Zhang, Ruibin Yuan, Ge Zhang, Linyang Li, Hang Yan, Jie Fu, Tao Gui, Tianxiang Sun, Yu-Gang Jiang, Xipeng Qiu
| Challenge: | Existing language models that use discrete representations for unified processing of various modalities are limited to text generation and do not include multimodal output. |
| Approach: | They propose a multimodal language model that utilizes discrete representations for unified processing of various modalities. |
| Outcome: | The proposed model can be trained stably without any alterations to existing models or training paradigms. |
AliGATr: Graph-based layout generation for form understanding (2024.findings-emnlp)
Copied to clipboard
| Challenge: | State of the art forms understanding models often rely on poorly calibrated output probabilities and low performance on relation extraction tasks. |
| Approach: | They propose a graph-based model that uses a generative objective to represent complex grid-like layouts that are often found in forms. |
| Outcome: | The proposed model performs better on the KIE and RE tasks and is more accurate than existing models. |